Student Success

Examining Quantitative Metrics

Dr Philip Leftwich

About me

Associate Professor in Data Science and Genetics at the University of East Anglia.


Academic background in Behavioural Ecology, Genetics, and Insect Pest Control.


I teach Genetics, Programming, and Statistics

UEA logo

What to expect during this talk

A short talk on using quantitative data to measure student success


I hope you end up with more questions than answers after this talk!


Schitts Creek questions gif

Source: giphy.com

Questions

  • What, if any, quantitivative measures predict student success?


  • Can we use these measures to intervene earlier with students who would otherwise not submit work?


  • Do Quantitative measures align with student perceptions of success/effort?


  • Do Quantitative measures align with our perceptions of success/effort?

Background

  • A cohort of Biological sciences students


  • Data is from a second year module on statistics and programming
    • 1 x Weekly lecture
    • 1 x Weekly Workshop

  • 69 students opted-in to measures of:

    • In-person attendance
    • Logged work hours
    • Participation in weekly assignments/tests
    • Summative assignment at the end of term

Descriptive Statistics

In-person attendance

Weekly tasks completion rates

Effort hours

Correlations

  • Coursework Days - Time between coursework started and submitted

  • Learning Hours - Time on system before coursework released

  • Coursework Hours - Time on system after coursework released

  • Attendance - In-person attendance

  • Weekly assignment - Completion of weekly tasks/tests

Predictive Outcomes

Characteristic Beta 95% CI1 p-value
(Intercept) 66 63, 68 <0.001
Coursework Hours 0.07 -0.15, 0.30 0.5
Coursework Days 0.81 0.58, 1.0 <0.001
Attendance -1.8 -6.1, 2.4 0.4
Coursework Hours:Attendance 0.30 0.01, 0.60 0.045
1 CI = Confidence Interval

  • Mean coursework mark 66%

  • Model accounts for 60% of variance in student attainment

  • Multiple candidate models were combined to produce an ensemble model that provides the most robust and reliable predictions

Non-submission

Probability of non-submission

  • 9/69 students did not submit coursework

  • Using a logistic regression model to separate coursework by submission/non-submission

  • Can we use quantitative measures to identify students at risk of non-submission?

Probability of non-submission

The first four weeks are a good indicator for risk of non-submission

Term 2

  • Continued weekly assignments

  • A course test in week four

  • Summative assignment at the end of term

Do early markers of engagement continue to predict student outcomes?

Applications

Black box - Machine learning

PROS

  • Maximum predictive ability

  • Identify and intervene with students

  • Can be trained on a wide range of data

CONS

  • Opaque

Identify key variables

PROS

  • Transparent

  • Engages students with decision making

CONS

  • Less accurate

  • More difficult to implement?

Where next

  • Studies to compare Quantitative and Qualitative measures of student success

  • Evaluate student perceptions of requirements or indicators of “academic success”

  • Larger scale data training to build predictive models

Wrap-up

  • Increasing amounts of data available on student participation and engagement

  • Student engagement is not as simple as “In-person attendance”

  • Quantitative markers CAN be used as predictors of academic achievement


philip-leftwich

@ecoevo.social@PhilipLeftwich

@PhilipLeftwich

PhilipLeftwich

philip.leftwich.github.io

p.leftwich@uea.ac.uk philip.leftwich.github.io philip-leftwich